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FOREWORD 


The  Forecasting  and  Allocation  of  Army  Recruiting  Resources  Study- Sequential 
Hierarchical  Allocation  of  Resource  Elements  (FAARRS-SHARE)  system  is  used  by  U.S.  Army 
Recruiting  Command  (USAREC)  to  facilitate  the  efficient  allocation  of  active  component 
recruiting  resources.  The  Army  also  has  a  need  to  efficiently  allocate  the  reserve  component 
recruiting  resources.  This  study  explores  the  feasibility  of  using  for  the  reserve  component  the 
FAARRS-SHARE  methodologies  developed  for  the  active  component.  A  reserve  component 
FAARRS-SHARE  system  would  provide  USAREC  with  a  means  of  analyang  the  impact  of 
recruiting  resources  on  accessions,  the  ability  to  estimate  the  resources  required  to  reach  specific 
accession  levels,  as  well  as  the  ability  to  do  analysis  of  the  trade-off  between  recruiters  and 
advertising. 

The  Selection  and  Assignment  Research  Unit  of  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences’  Manpower  and  Personnel  Research  Division  conducted  this  study 
under  Task  1331,  “Personnel  Policy  Analysis,”  as  part  of  the  Study  and  Analysis  (6.6)  program. 
The  Directorate  of  Military  Personnel  Management  requested  this  study.  The  findings  of  this 
study,  which  were  provided  to  USAREC,  will  support  the  decision  on  the  development  of  a 
reserve  component  version  of  the  FAARRS-SHARE  system. 


ZITA  M.  SIMUTIS 
Deputy  Director 
(Science  and  Technology) 


EDGAR  M.  JOHNSON 
Director 
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FEASroiLITY  STUDY  OF  A  FAARRS-SHARE  METHODOLOGY  FOR  THE  U  S.  ARMY 
RESERVE 

EXECUTIVE  SUMMARY 


Research  Requirements: 

The  Army’s  recruiting  resources  allocation  decisions  are  critical  in  an  era  of  diminishing 
resources.  The  Army  s  recruiting  community  needed  tools  to  aid  them  in  the  efficient  and 
effective  allocation  of  their  limited  resources.  A  methodology  is  required  for  both  the  active  and 
reserve  components  that  forecasts  high-quality  contracts  given  resource  levels,  estimates  the 
resources  required  to  achieve  a  given  level  of  accessions,  and  measures  the  effects  and 
interactions  of  management  decisions.  The  Forecasting  and  Allocation  of  Army  Recruiting 
Resources  Study-Sequential  Hierarchical  Allocation  of  Resource  Elements  (FAARRS-SHARE) 
system  provides  this  capability  for  the  active  component.  There  is  a  need  for  a  similar  system  for 
the  reserve  component.  This  system  vvill  provide  the  Army  with  information  it  needs  to  make 
critical  decisions  in  an  environment  of  diminishing  recruiting  resources. 

Procedure: 

A  new  data  envelopment  analysis  (DEA)  approach  that  provides  simultaneously  efficiency 
evaluations  and  sensitivity  or  robustness  measures  are  described.  Building  on  this  DEA  approach, 
new  formulations  that  combine  goal  programming  and  discriminant  function  techniques  for 
estimation  of  an  approximate  empirical  production  for  U.S.  Army  Reserve  recruiting  are 
presented.  Solution  procedures  are  then  described.  A  quarterly  fiscal  year  1993  database 
containing  factors  that  effect  the  production  of  nonprior  service  and  prior  service  reserve 
accessions  is  used  in  the  DEA  analyses.  These  data  include  the  following  input  factors:  the 
number  of  reserve  recruiters,  the  reserve  mission,  unemployment  rate,  local  expenditures,  target 
population,  and  reserve  gross  rating  points  for  television,  cable,  radio,  and  print.  Estimation  of 
empirical  production  function  was  performed  for  the  aggregate  year-based  efficiency  results. 

Findings: 

Estimation  of  the  empirical  production  function  produces  an  elasticity  of  .83  for  mission 
and  .  14  for  unemployment  with  all  other  factors  having  negligible  impacts  on  the  production  of 
nonprior  service  and  prior  service  reserve  accessions.  In  excluding  reserve  mission  as  a  factor  in 
the  model,  the  effects  of  unemplo5mient,  local  expenditures,  and  national  advertising  are 
approximately  .10  with  the  recruiter  elasticity  between  .70  and  .76.  The  estimation  device 
employed  here  is  sound.  However,  the  accuracy  of  the  estimation  results  depends  on  there  being 
a  sufficient  number  of  observations.  Improvement  in  estimation  results  can  be  achieved  with 
increases  in  the  number  of  observations. 


Utilization  of  Results: 


These  results  can  be  used  as  the  foundation  for  building  an  empirical  production  function 
for  Army  reserve  recruiting.  In  addition,  the  new  methodologies  presented  can  be  utilized  to 
improve  the  Army  active  component  FAARRS-SHARE  system. 
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FEASIBILITY  STUDY  OF  A  FAARRS-SHARE  METHODOLOGY  FOR  THE  U.S.  ARMY 

RESERVE 

Introduction 


This  report  details  research  into  a  new  methodology  for  the  potential  development  of  a 
Forecasting  and  Allocation  of  Army  Recruiting  Resources  Study-Sequential  Hierarchical 
Allocation  of  Resource  Elements  (FAARRS-SHARE)  system  for  U.S.  Army  Reserve  (USAR) 
use.  The  present  work  is  motivated  by  the  developments  originally  presented  in  Chames,  Golany, 
Pitaktong,  and  Rousseau  (1991)  and  in  enhanced  form  in  Chames,  Kress,  Golany,  Pitaktong,  and 
Rousseau,  Semple,  Song,  and  Zhou  (1992)  but  differs  from  them  in  several  significant  respects. 
See  also  Thomas  (1990). 

The  purpose  of  the  original  (FAARRS-SHARE)  research  was  to  develop  a  simple,  rapid 
response  methodology  and  accompanying  software  for  forecasting,  allocation  and  evaluation  of 
annual  Active  Army  recruiting  resources  at  the  aggregate  headquarters  Department  of  the  Army 
(HQD  A)  level.  Such  a  system  was  required  in  order  to  implement  a  multiyear  program  of 
desired  accessions,  or  to  quickly  (within  72  hours)  evaluate  for  congressional  information  the 
effects  of  sugg^sfed  changes  in  accessions  or  in  resource  availabilities  across  such  a  multiyear 
program.  Successful  completion  of  the  initial  research  and  development  of  software  was 
achieved  in  a  six  month  period  during  fiscal  year  (FY)  1991,  and  the  initial  SHARE  system  has 
since  been  in  practical  use  at  both  Office  of  Deputy  chief  of  Staff  of  Personnel  (ODCSPER)  and 
U.S.  Army  Recmiting  Command  (USAREC). 

As  a  result  of  this  usage,  additional  features  and  capabilities,  not  previously  requested  by 
ODCSPER,  were  recognized  as  desirable  and  subsequently  incorporated  in  the  enhanced  version. 
In  addition,  the  methods  and  procedures  developed  in  FAARRS-SHARE  were  viewed  as 
potentially  having  wider  applicability  than  just  to  the  Active  Component  (AC). 

The  present  research  re-examines  the  underlying  models  of  the  FAARRS-SHARE  system 
and  begins  investigation  of  a  new,  more  appropriate  methodology  for  developing  such  a  system 
for  Army  Reserve  use.  The  most  recent  developments  in  sensitivity/stability  analysis  in  data 
envelopment  analysis  (DEA)  are  researched  as  the  basic  level  of  analysis.  Building  on  this,  other 
new  and  novel  formulations  are  developed  which  combine  goal  programming  and  discriminant 
function  techniques  for  estimation  of  an  approximate  empirical  production  function  for  U.S. 
Army  Reserve  recruiting. 

The  DEA-based  micro  analysis  is  a  non-parametric  methodology  that  requires  no  a  priori 
model  specification;  the  goal  programming-discriminant  function  procedures  involve  analytical 
or  parametric  solutions  —  hence  we  call  this  a  "semi-parametric"  method. 

This  report  is  organized  as  follows.  The  following  two  sections  provide  the  necessary 
background  on  FAARRS-SHARE  and  a  discussion  of  its  applicability  to  the  Army  Reserve  that 
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lead  to  the  development  of  our  new  approach.  The  fundamental  DEA  analyses  are  discussed  in 
the  fourth  section,  followed  by  our  new  methods  for  empirical  production  function  estimation. 
The  sixth  section  discusses  data  issues  associated  with  implementation  of  our  procedures  using 
FY  1993  data,  and  a  seventh  section  presents  our  results.  A  final  section  summarizes  our  main 
findings  and  points  towards  some  further  avenues  of  research. 


Background 

As  discussed  in  Chames  et  al.  (1991)  and  Chames  et  al.  (1992)  the  Army  recruiting 
battalion  is  the  "lowest"  management  element  that  has  direct  control  over  primary  resource 
decisions  that  lead  to  "production"  of  contracts  for  Army  service.  Moreover,  since  the  recent 
downsizing  trend  in  Army  strength  coupled  with  drastic  reductions  in  defense  budgets  required 
forecasting  across  much  lower  accession  levels  than  in  previous  recruitment,  better  instruments 
needed  to  be  developed  that  would  be  consistent  with  USAREC  command  and  control 
recruitment  management  and  not  conflict  with  USAREC  experience.  Meeting  these 
requirements  made  possible  the  new  "multiplicative  aggregation"  method  developed  in  Chames 
et  d.  (1991)  and  Chames  et  al.  (1992)  and  Thomas  (1990)  for  FAARRS- SHARE  which  permits 
one  to  go  not  only  from  battalion  to  total  command  but  also  from  quarterly  accessions  to  multiple 
quarters  and  to  yearly  accessions. 

TTie'FAARRS-SHARE  process  employs  a  multiplicative  Data  Envelopment  Analysis 
(DEA)  model  to: 

(a)  approximate  an  empirical  production  function; 

(b)  determine  the  most  consistently  best  performing  battalions  (BNs);  and 

(c)  achieve  therefrom,  by  a  constrained  weighted  least  absolute  value  method  ('goal 
enhancement'),  the  relative  rates  of  change  ('elasticities')  in  optimal  (so-called 
'technically  efficient')  performance  of  recruiting  resource  and  environmental 
elements. 

This  methodology  overcomes  several  problems  frequently  associated  with  traditional 
statistical  regression  approaches  to  estimation.  The  problem  of  model  mispecification  (i.e., 
incorrect  choice  of  inputs  or  outputs)  is  obviated  here  by  more  than  fifteen  years  experience  with 
USAREC  data  and  recruiting  activities.  A  second  potential  difficulty  concerns  the  robustness  of 
the  objective  function  criterion.  Here  we  use  weighted  least  absolute  value  rather  than  the  very 
sensitive  least  squares  or  maximum  likelihood  to  ensure  robustness.  A  third  problem  can  be  that 
of  collinearity  or  lack  of  independence  of  inputs  and  of  outputs.  Here  the  absolute  value  criterion 
plus  constraints  to  ensure  valid  specification  minimized  this  problem  compared  to  using  least 
squares  or  maximum  likelihood. 


2 


Potential  Applicability  To  Army  Reserve 


We  began  initial  investigations  of  what  would  likely  be  involved  in  extending  the 
methodology  to  USAR  recruiting.  Here  we  sought  to  identify  general  data  requirements, 
determine  data  availability  and  integrity,  and  familiarize  ourselves  with  the  special  characteristics 
and  nuances  of  USAR  recruiting  which  could  become  problem  areas.  We  can  summarize  our 
findings  as  follows. 

Generally,  the  same  kinds  of  core  resources  are  required  to  secure  quality  Reserve  recruits 
as  are  for  AC  recruits,  but  their  differential  effects  may  be  quite  different  in  the  two  cases. 
Moreover,  we  can  expect  additional  factors,  not  so  relevant  in  the  AC  case,  to  have  an  impact  in 
the  Reserve  situation.  For  example,  the  role  of  potential  differentials  between  civilian  and 
military  pay  in  the  market  will  need  to  be  examined  closely,  and  the  issue  of  market  proximity  to 
USAR  centers  will  have  to  be  explored. 

Other  issues  further  complicate  matters.  Several  DEA  analyses  would  likely  be  necessary 
in  order  to  determine  whether  the  AC  (and  perhaps  the  National  Guard)  should  be  treated  as 
competition  in  Reserve  recruiting.  Competition  from  the  other  reserve  components  is  also  an 
important  consideration,  but  what  little  information  there  is  on  competitive  activity  is  of  poor 
quality.  For  example,  the  presence  and  extent  of  competitive  activity  at  the  battalion  level  is 
unknown,  and  it  is  unclear  whether  the  other  service  reserve  components  can  provide  that 
information. 

Many  of  the  dollar  expenditures  on  automation,  communications,  the  delay  entry  program 
(DEP),  facilities,  vehicles,  advertising,  etc.  incurred  by  higher  echelons  of  management  also 
support  the  USAR,  so  developing  an  appropriate  allocation  of  cost  in  each  funding  area  will  be  a 
major  factor. 

In  addition  to  such  considerations,  there  is  also  the  basic  question  of  the  continued 
validation  of  the  mathematical  procedures  underlying  FAARRS-SHARE.  This  is  briefly 
discussed  next. 

Appropriateness  of  the  FAARRS-SHARE  Methodology 

As  innovative  as  the  original  FAARRS-SHARE  models  were,  recent  developments  in 
DEA  and  related  techniques  for  empirical  production  function  estimation  suggest  that  significant 
improvements  to  the  methodology  are  now  possible. 

These  improvements  begin  at  the  most  fundamental  level  with  superior  techniques  for 
conducting  the  basic  DEA  analyses  and  determining  the  most  robustly  efficient  recruiting 
battalions  which  will  drive  estimation  of  the  empirical  production  function.  These  new  methods, 
which  incorporate  notions  of  sensitivity  or  stability  in  DEA  efficiency  classifications,  are 
discussed  in  the  next  section. 

Equally  significant  are  the  new  formulations  for  production  function  estimation  which 
combine  in  a  novel  way  ideas  from  goal  programming  with  those  from  the  use  of  discriminant 
functions.  These  procedures  are  detailed  below. 
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DEA  Analyses 


Data  Envelopment  Analysis  (DEA)  performed  at  the  recruiting  battalion  level  is  the  basic 
building  block  for  our  proposed  methodology.  However,  unlike  the  previous  FAARRS-SHARE 
research,  the  fundamental  DEA  analyses  were  here  conducted  using  new  formulations  that 
provide  simultaneously  the  required  efficiency  evaluations  in  addition  to  important  measures  of 
the  sensitivity  or  robusmess  of  the  efficiency  classifications  (see  references  Chames,  Rousseau, 
and  Semple  (1995)  and  Rousseau  and  Semple  (1995)). 

A  frequent  criticism  of  the  DEA  results  is  that  they  present  only  a  snapshot  of  a  unit's 
situation,  disregarding  the  effects  of  errors  in  the  data,  both  real  and  induced,  and  the  transitory 
namre  of  efficiencies  in  general.  Lost,  therefore,  is  any  indication  of  whether  the  classification  is 
fleeting  or  robust.  Without  this  sensitivity  information,  therefore,  findings  can  be  distorted  when 
marginally  efficient  or  inefficient  units  are  distinguished  solely  on  the  basis  of  their 
classification.  To  account  for  this  additional  factor  without  introducing  distributional 
assumptions  on  the  data,  we  employ  the  particular  sensitivity  analysis  discussed  in  Chames  et  al. 
(1995)  and  illustrated  in  Rousseau  et  al.  (1995). 

In  this  new  sensitivity  approach,  the  unit's  observed  input-output  levels  serve  as  the 
center  of  a  ball  (or  cell)  in  the  multi-dimensional  input-output  space.  The  largest  ball  for  which 
the  interior  contains  strictly  classification  preserving  levels  (with  respect  to  the  remaining 
unperturbed  reference  group)  is  then  computed.  We  call  the  radius  of  this  maximal  ball  the 
radius  of  classification  preservation  (RCP)  and  interpret  it  as  a  measure  of  the  robustness  of  the 
DEA  classification.  Moreover,  when  either  the  /]  or  /-infinity  (Tchebycheff)  norm  is  used  to 
describe  the  ball,  the  computations  involve  solving,  at  worst,  a  finite  number  of  related  linear 
programs. 

It  is  important  to  realize  that  the  values  of  these  radii  also  answer  a  fundamental  question: 
what  is  the  minimum  distance  the  analyzed  unit  must  be  moved  to  alter  its  classification? 
Therefore,  like  the  efficiency  score,  which  is  based  upon  the  principle  of  maximizing  the 
separation  between  a  unit  and  the  reference  points  that  dominate  it,  the  radius  of  classification 
preservation  is  motivated  by  a  particular  extremal  property.  However,  unlike  the  efficiency 
score,  the  radii  can  be  used  to  discriminate  among  efficient  units  and  reveal  important  differences 
among  inefficient  units. 

The  linear  program  (TR)  (T  for  Tchebycheff.  R  for  radius)  below  provides  the  necessary 
computations  for  efficient  and  inefficient  units  alike  (with  index  k) 
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The  superscripted  index  indicates  that  the  associated  column  has  been  omitted  from  the  matrices 
of  observed  outputs  (Y)  and  inputs  (X),  without  regard  to.  or  prior  knowledge  of.  the  test  unit's 
classification.  The  magnitude  of  the  optimal  value  represents  the  minimum  absolute  change 
needed  to  place  the  test  unit  infinitesimally  close  to  reclassification.  The  absolute  value  of  the 
optimal  value  to  (TR)  is  referred  to  as  the  unit's  Tchebycheff  radius  of  classification  preservation. 

An  alternative  approach  is  to  compute  the  minimum  relative  change  needed  to  effect 
reclassification.  In  this  case,  the  vectors  es  and  em  are  replaced  by  the  vectors  yk  and  xk, 
respectively  in  (TR).  The  ball  is  no  longer  associated  with  the  Tchebycheff  norm  but  rather  a 
generalized  Tchebycheff  norm,  which  includes  weights  on  the  individual  dimensions  of  the 
vector.  If  the  test  unit's  inputs  and  outputs  are  used  as  the  source  of  the  weights,  then  the 
formulation  which  computes  the  percentage  change  needed  simultaneously  in  all  inputs  and 
outputs  to  effect  reclassification,  for  efficient  and  inefficient  units  alike,  is  given  by  the  linear 
program  (GTR). 


Min  -a 
Subject  to 

Y(k)A,-s^+a>^-a”y^ 

X(k)^  -  s'"  +  -Oi~Xi^ 

X„  s'',s~,a*,a~  >0. 

The  absolute  value  of  the  optimal  value  to  (GTR)  will  be  referred  to  as  the  unit's  generalized 
Tchebycheff  radius  of  classification  preservation. 

At  least  one  input  or  output  dimension  must  be  worsened  by  more  than  the  computed 
RCP  to  achieve  reclassification:  worsening  every  input  and  output  simultaneously  by  more  than 
this  value  does  bring  about  reclassification. 

Our  interest  in  (TR)  and  (GTR)  is  not  limited  to  their  obvious  computational  advantages. 
The  optimal  value  to  each  provides  a  more  comprehensive  evaluation  of  the  unit  than  a  simple 
efficiency  score.  The  sign  of  the  optimal  value  indicates  the  classification  of  the  test  unit:  a 
negative  sign  identifies  inefficient  units,  a  positive  sign  identifies  efficient  units.  Therefore,  (TR) 
and  (GTR)  are  stand-alone  DBA  models  with  the  advantage  that  efficiency  classifications  and 
sensitivity  information  are  computed  simultaneously. 

Equally  important  is  the  magnitude  of  the  optimal  value  since  it  addresses  the  robusmess 
of  the  classification.  If  interest  centers  on  the  likelihood  of  a  classification  maintaining  its 
current  status  in  the  future,  then  the  magnitude  of  the  optimal  value  becomes  an  important 
indicator. 

The  RCPs  are  further  important  since  they  can  be  used  to  distinguish  among  units  with 
identical  classifications.  This  will  be  particularly  relevant  in  the  research  of  the  next  section 


(GTR) 


=  1 
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where  a  satisfactory  division  of  recruiting  battalions  into  efficient  and  inefficient  groups  forms 
the  basis  for  estimation  of  the  empirical  production  function. 

These  new  DEA  formulations  have  several  other  advantages: 

*  There  is  no  need  to  generate  and  then  justify  specific  efficiency  scores  obtained  from  a 
particular  DEA  model. 

•  The  programs  can  be  implemented  using  any  LP  solver  without  recourse  to  any  existing 
DEA  code. 

*  Subsequent  analysis  of  the  RCPs  may  become  important. 

•  Having  programs  (TR)  and  (GTR)  as  a  point  of  departure  may  help  suggest  and 
motivate  other  applications  of  this  type. 

Much  of  classical  single-output  production  theory  is  centered  on  fitting  a  production 
function  to  the  observed  data,  otherwise  known  as  the  econometric-regression  approach.  After  a 
function  has  been  determined  with  a  suitable  fit,  individual  units  are  assessed  by  measuring  the 
difference  between  their  observed  output  and  the  value  computed  by  the  theoretical  production 
function.  This  difference  is  the  error  or  'residual'  for  the  unit  and  represents  the  distance 
separating  observed-froih  theoretically  efficient  production. 

■’  *  -  i  ''- 

*  In  the  ?norocomplicated»multi-output  case  of  DEA,  the  production  function  is  replaced  by 
an  input-output  correspondence,  also  determined  by  the  sample  data,  which  defines  Pareto 
efficient  production.  Because  the  radius  of  classification  preservation  measures  the  distance 
separating  each  unit  from  reclassification,  it  constitutes  a  generalization  of  the  residual 
encountered  in  the  single  output  production  case.  The  importance  of  these  residuals  for  future 
research  and  analysis  will  be  discussed  further  in  our  Summary. 


Empirical  Production  Function  Estimation 

The  ability  to  represent,  even  in  approximate  form,  the  recruiting  input-output  relations 
by  means  of  a  frontier  function  is  an  important  step  in  using  historical  data  (observed  inputs  and 
outputs)  for  current  and  future  managerial  planning.  For  example,  resource  allocation  and 
reallocation  across  recruiting  battalions  can  be  based  on  the  frontier  function.  Questions  of 
priorities,  relative  importance  of  production  factors  and  goal  setting  can  all  be  addressed  using 
this  function. 

This  section  adapts  to  the  DEA  context  a  goal  programming  model  developed  by  Freed 
and  Glover  (1981)  as  a  general  purpose  instrument  for  discriminating  among  specified  groups  of 
observations.  For  a  complete  discussion,  see  Golany  and  Yu  (1995)  and  references  therein.  The 
data  for  the  model  include  a  vector  of  observed  values  for  each  entity  (here  a  recruiting  battalion) 
and  its  association  with  one  of  several  groups.  In  the  present  case  we  consider  only  two  groups: 
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efficient  and  inefficient  battalions.  To  associate  each  evaluated  battalion  with  either  the  efficient 
(E)  or  inefficient  group,  we  apply  the  (TR)  or  (GTR)  models  of  the  previous  section. 

We  then  apply  a  variant  of  the  model  developed  by  Freed  and  Glover  to  estimate  the 

parameters  of  the  discriminant  function.  The  original  model  considered  Aj.  j  =  I . n 

observations  organized  into  groups  Gk,  k  =  1 . g.  A  pair-wise  (i.e.,  g  =  2)  variant  of  the  model 

was  formulated  as; 


Min 

subject  to 

AjX-aj  +6j=b,  j  eG^ 

AjX —OL  j  +b  j  =  b,  J  G  Gi 

a^.,5^.  >0. 


where  X  represents  a  linear  predictor  (or  a  weighting  scheme),  b  is  a  scalar  breakpoint  between 
the  two  groups,  aj  are  misclassification  deviations  that  are  minimized  and  6j  are  desired 
deviations  which  are  maximized  in  the  objective  function  which  uses  hj  and  cj  as  weights. 


The  model  selects  a  separating  hyperplane  AX  =  b  in  a  way  that  attempts  not  only  to  have 
all  the  units  in  one  group  above  it  and  all  units  in  the  other  group  below  it  (by  minimizing  the  aj 
deviations)  but  also  to  'push'  the  units  in  the  respective  groups  as  far  away  as  possible  from  the 
hyperplane  (by  maximizing  the  5j  deviations)  so  as  to  provide  the  sharpest  possible  distinction. 


The  input-output  relations  in  DEA  are  characterized  by  a  piece-wise  concave  empirical 
function,  so  the  model  above  must  be  transformed  to  convert  the  concave  relations  assumed  for 
DEA  into  a  hnear  function.  Here  we  use  a  Translog  transformation  which  has  the  advantages  of 
being  general  and  allowing  for  synergy  effects  among  the  inputs  to  be  formulated  explicitly. 
Hence  we  define  the  observations  Aj  as  follows: 


log(T,). 

-lOg(Xy). 

-10g(Xy)IOg(X,p, 


r  =  l,...,s 
i  =  l,...,m 
i,k  = 


The  groups  are  defined  as 

G2  =  E  (the  set  of  efficient  battalions) 
Gi  =  I  (the  set  of  inefficient  battalions) 
and  the  linear  predictor  is  given  by 
X  =  (^i,v). 

Thus  our  Translog  function  can  be  expressed  as 
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Xlog(Y„)n,  =  3  +5;><>8(Xs)v,  j 

r  i  i  k 

Without  loss  of  generality  we  normalize  the  v  variables  to  prevent  an  'all- zero'  solution. 
This  results  in  the  following  general  model  (G): 

Min  Shija,  -Xc., 5,  -Xc:;5j 

jel  jeE  jGl  jeE 

Subject  to 

Xlog(Y,)n,  -Xlog(X,i>u,  -XXlog<(X,)log(X,)u.  -a;  +5,  = 

r  i  i  k 

Xlog(Y,)n,  -Xlog(X„)«,  -XXlog((X,)log(X„)u.  -a,  +5,  = 

r  i  i  k 

i  i  k 


P*  y  e  / 
P.  J^E 


(G) 


Solution  Procedure 


Based  on  our  development  above  we  now  propose  the  following  procedure  to  identify  the 
parameters  of  a  Translog  function  which  can  serve  as  a  surrogate  for  the  empirical  production 
function  in  recruiting. 

Step  1.  Solve  (TR)  or  (GTR)  and  associate  the  recruiting  battalions  with  the  sets  E  and  I. 

Step  2.  Solve  the  following  variant  of  the  general  model  (G)  above. 

Min  z 
Subject  to 

Xlog(Y,)tl,-Xlog(X,)u,-2;Xlog((X„)log(Xj|)u.  <p  jel 

r  i  i  k 

Xlog(Y,)n,  -Xlog(X,)u,  -XX‘‘>8((X,i)log(X,)u.  -a,  +5,.  =  P,  y  s  £ 

r  i  i  k 

aj  <z,  jeE 

f  i  k 

aj,dj  >0,y/. 


Drop  from  consideration  all  battalions  in  E  with  z  =  aj*. 

Repeat  Step  2  until  z  =  0  or  lEI  <  p,  that  is.  until  no  further  misclassification  errors  are 
observed  for  the  battalions  in  E  or  until  the  number  of  remaining  battalions  in  E  has  reached  a 
prescribed  minimal  level  p. 
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Step  3.  Solve  the  following  variant  of  the  general  model  (G)  above. 

Min  2^(a^.+5j.) 

jeE 

Subject  to 

2log<Y,)n,-Xlog(X,)u,-2;ilog((X,)log(X„)l).  sp  jel 

r  i  i  k 

2;iog(Yj)H,  -Xlog(X,)u,  -XS'“8«Xs)‘‘>8(X,i)u.  -a,  +  6;  =  P,  y  €  £ 

r  i  i  k 

i  i  k 

aj,bj  >  o,y/. 

A  key  to  understanding  the  procedure  above  is  the  realization  that  DEA,  being  a  relative 
efficiency  method,  is  more  likely  to  misclassify  as  efficient  battalions  which  actually  are 
inefficient  according  to  an  absolute  standard  rather  than  the  reverse  case.  When  DEA  identifies  a 
unit  as  inefficient,  this  means  that  evidence  has  been  found  that  other  units  dominate  it.  On  the 
other  hand,  when  DEA  identifies  a  unit  as  efficient  this  means  only  that  no  such  evidence  of 
domination  has  been  found  in  the  observed  data,  but  this  does  not  imply  that  this  unit  is  indeed 
necessarily  efficient  with  respect  to  the  unknown  production  function. 

Thus  the  purpose  of  Step  2  above,  which  may  be  repeated  several  times,  is  to  drop 
battalions  in  set  E  from  further  consideration  if  they  are  suspected  of  being  misclassified  into  E. 
In  Step  2  we  identify  in  each  iteration  the  battalions  that  are  associated  with  the  largest 
misclassification  error  and  drop  them  from  set  E.  This  step  is  repeated  until  no  further 
misclassification  errors  are  observed  for  the  battalions  in  E  or  until  the  number  of  remaining 
battalions  in  E  has  reached  a  prescribed  minimal  level  p. 

Step  3  continues  to  enforce  the  separation  that  was  attained  earlier  (in  particular,  it  allows 
no  mi.sclassification  errors  for  battalions  in  I)  while  attempting  to  fit  a  hyperplane  through  the 
reduced  set  of  efficient  battalions.  In  that  context.  Step  3  can  be  viewed  as  a  Minimum  Absolute 
Deviation  (MAD)  regression  applied  to  E  with  side  constraints  related  to  battalions  in  I. 

Refinements 


Our  solution  procedure  can  be  refined  by  directly  incorporating  the  sensitivity 
information  provided  by  (TR)  or  (GTR). 

The  individual  radii  of  classification  preservation  can  be  used  to  form  the  weights  hij, 
b2j>  cij  and  C2j  in  the  objective  function  of  the  general  model  (GM).  Thus  the  desirable 
deviations  5j  and  the  undesirable  deviations  aj  can  be  given  weights  that  reflect  the  robustness  of 
battalion  j's  classification  as  efficient  or  inefficient. 

Alternatively,  the  radii  of  classification  preservation  can  be  used  to  partition  the 
battalions  into  finer  groupings.  The  efficient  battalions  might  be  partitioned  into  sets  E^,  E^. 
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E®  according  to  the  magnitudes  of  the  radii,  for  example,  might  contain  the  upper  decile  and 
E®  the  lowest  decile.  Likewise,  the  inefficient  battalions  might  be  grouped  into  (upper  ten 
percent), E  (lowest  ten  percent).  The  groupings  in  which  we  would  have  the  most  confidence 
are  E^  (most  robustly  efficient)  and  (most  robustly  inefficient).  The  potential  for 
misclassification  is  greatest  with  sets  E®  and  E.  Accordingly,  these  latter  four  groups  of 
battalions  might  be  singled  out  for  special  treatment  (in  different  ways  and  for  different  reasons) 
in  the  model  constraints.  A  variety  of  other  possibilities  are  also  available. 


Data  Issues 

Following  is  a  list  of  outputs  and  inputs  used  in  our  initial  analyses.  Data  were  available 
for  each  quarter  of  FY  1993.  In  that  year  there  was  a  42  battalion  structure. 

Outputs 

Non-Prior  Service  (NPS)  contracts  -  male  plus  female,  all  categories 

Prior  Service  (PS)  contracts  -  male  plus  female,  all  categories 

Inputs 

tiSAR  Recruiters 

Reserve  Mission 

Unemployment  Rate 

USAR  Local  Expenditures 

Target  Population 

USAR  Aggregate  Advertising  Gross  Rating  Points  (GRPs) 

TV 

Cable 

Radio 

Print 

Several  other  input  factors  were  considered  but  could  not  be  used  due  to  lack  of  available 
or  reliable  data.  Significant  among  these  were  measures  of  competitive  activity  (e.g.,  from  the 
national  guard)  and  the  civilian/military  wage  differential.  Regarding  the  latter,  there  are 
potential  difficulties  in  its  measurement.  Determining  the  appropriate  civilian  occupational 
categories  is  one  issue;  combining  or  somehow  averaging  wage  rates  for  different  occupations  is 
another;  averaging  the  result  across  distinct  regions  in  a  battalion  is  yet  another.  Moreover,  no 
data  on  wage  rates  were  available  to  us. 

Even  for  the  inputs  used,  some  issues  are  still  unresolved.  For  example,  it  is  not  clear 
that  the  target  population  age  group  is  precisely  the  same  as  that  for  which  advertising  GRPs  are 
reported.  The  input  actually  used  in  the  DEA  is  impressions  (=  GRPs  x  target  population),  so 
this  potential  mismatch  may  be  detrimental  to  the  analysis.  In  addition,  an  unemployment  rate 
for  the  target  population  was  not  available  to  us  and  a  general  unemployment  rate  had  to  suffice 
as  a  surrogate. 


10 


In  these  preliminary  efforts  we  are  seeking  to  determine  the  appropriateness  of  our 
general  procedures  for  evaluation  of  Reserve  recruiting  activities.  Consequently,  for  the  early 
DEA  runs  we  chose  to  use  aggregate  advertising  GRPs  as  an  input  rather  than  also  attempt  to 
unravel  the  media  mix  issues  through  having  separate  inputs  for  each  medium. 


Results 


Efficiency  Results 

The  DEA  runs  were  carried  out  using  (GTR)  for  each  of  the  four  quarters  of  FY  1993  and 
on  aggregate  data  for  the  year  as  a  whole.  The  efficiency  results  are  summarized  in  Tables  Al- 
A5  in  the  Appendix.  Recall  from  the  DEA  Analyses  section  that  the  sign  of  a  battalion's  alpha 
value  (column  three  of  each  table)  indicates  its  efficiency  classification:  positive  alpha  indicates 
efficient,  negative  indicates  inefficient. 

Recall  also  that  the  absolute  value  of  alpha  represents  the  robustness  of  the  classification. 
Accordingly,  in  quarter  1  of  FY  1993  (see  Table  Al)  Battalion  17  would  have  required  across- 
the-board  changes  to  inputs  and  outputs  in  excess  of  22%  in  order  to  have  been  classified  as 
inefficient,  while  Battahon  42  would  have  required  changes  of  only  5%.  Likewise,  inefficient 
Battalion  18  was  only  marginally  so,  requiring  simultaneous  improvements  in  inputs  and  outputs 
of  only  1%  in  order  to  effect  reclassification.  However,  Battalion  39  would  have  needed  ;  • 
improvements  of  30%  to  have  been  reclassified  as  efficient. 

Also  shown  in  Tables  A1-A5  are  the  "proportion  of  mission  achieved"  and  "write  rate" 
for  each  battalion  and  for  the  Reserve  recruiting  command  as  a  whole,  by  quarter  and  annually. 

In  each  case,  the  figures  are  given  for  Non-Prior  Service  contracts  and  Prior  Service  contracts  as 
well  as  in  aggregate.  These  two  factors  may  be  viewed  as  measures  of  recruiting  effectiveness, 
not  to  be  confused  with  recruiting  efficiency.  We  will  return  to  this  in  more  detail  later. 

Summary  Table  1  indicates  that  there  are  both  temporal  and  geographic  differences  in 
recruiting  performance.  In  FY  1993  fifteen  battalions  were  efficient  in  quarter  1.  sixteen  in 
quarter  2,  twenty  in  quarter  3,  eleven  in  quarter  4,  and  fourteen  battalions  were  efficient  over  the 
aggregate  year.  Four  battalions  (Battahon  2.  Battalion  14,  Battalion  36,  Battalion  38)  were 
efficient  in  all  four  quarters  and  in  aggregate  for  the  year  as  a  whole. 

A  further  five  battalions  (Battalion  17,  Battalion  22,  Battalion  27,  Battalion  35,  Battalion 
42)  were  efficient  in  three  of  the  four  quarters,  but  only  the  middle  three  were  also  efficient  on 
an  annual  basis  (although  the  classification  of  both  Battalion  17  and  Battalion  42  was 
indeterminate,  i.e.,  with  an  annual  alpha  value  of  zero).  Seven  more  battalions  (Battalion  4, 
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Table  1 


Efficient  Battalions  FY  1993 


BN 

Ql 

Q2 

Q3 

Q4 

FY 

Battalion 

1 

Battalion 

2 

X 

X 

X 

X 

X 

Battalion 

3 

Battalion 

4 

X 

X 

X 

Battalion 

5 

X 

Battalion 

6 

X 

X 

X 

Battalion 

7 

Battalion 

8 

Battalion 

9 

X 

X 

X 

Battalion 

10 

X 

X 

X 

Battalion 

11 

X 

X 

X 

Battalion 

12 

Battalion 

13 

X 

X 

X 

Battalion 

14 

X 

X 

X 

X 

X 

Battalion 

15 

X 

Battalion 

16 

X 

Battalion 

17 

X 

X 

X 

Battalion 

18 

X 

Battalion 

19 

X 

Battalion 

20 

Battalion 

21 

X 

X 

Battalion 

22 

X 

X 

X 

X 

Battalion 

23 

X 

Battalion 

24 

Battalion 

25 

Battalion 

26 

Battalion 

27 

X 

X 

X 

X 

Battalion 

28 

X 

Battalion 

29 

X 

Battalion 

30 

Battalion 

31 

X 

X 

Battalion 

32 

Battalion 

33 

X 

Battalion 

34 

X 

X 

Battalion 

35 

X 

X 

X 

X 

Battalion 

36 

X 

X 

X 

X 

X 

Battalion 

37 

X 

X 

X 

Battalion 

38 

X 

X 

X 

X 

X 

Battalion 

39 

Battalion 

40 

X 

Battalion 

41 

X 

Battalion 

42 

X 

X 

X 

15 

16 

20 

11 

14 

12 


Battalion  6,  Battalion  9,  Battalion  10,  Battalion  11,  Battalion  13  and  Battalion  37)  were  efficient 
on  an  annual  basis  even  though  they  were  efficient  in  only  two  of  the  quarters.  Twelve  battalions 
were  never  efficient  in  any  quarter. 

The  42  recruiting  battalions  are  grouped  regionally  into  four  Brigades:  Brigade  A  has  13 
battalions.  Brigade  B  has  8,  Brigade  C  has  10,  and  Brigade  D  contains  1 1  battalions.  We  can 
think  of  a  given  battalion  in  a  specified  quarter  as  one  opportunity  for  the  battalion's  associated 
Brigade  to  excel.  Forming  the  ratio  of  such  efficiency  successes  to  the  Brigade's  number  of 
opportunities  provides  a  measure  of  Brigade  "efficiency  success"  for  each  quarter  and  annually. 
For  example,  in  quarter  1  Brigade  C  had  three  efficient  battalions  out  of  its  ten.  Hence  its 
efficiency  success  rating  in  quarter  1  is  0.30.  On  an  annual  basis  five  of  its  battalions  were 
efficient  so  its  annual  efficiency  rating  is  0.50.  The  Table  2  summarizes  this  information. 

Table  2 


Brigade  Efficiency  Success 


Q1 

Q2 

Q3 

Q4 

Mean 

FY93 

Brigade  A 

.62 

.462 

.462 

.231 

.404 

.308 

Brigade  B 

.50 

.625 

.625 

.50 

.563 

.50 

Brigade  C 

.30 

.20 

.50 

.30 

.325 

.50 

Brigade  D 

.182 

.273 

.364 

.091 

.227 

.091 

Command 

.357 

.381 

.476 

.262 

.369 

.333 

Brigade  B  is  consistently  best  by  this  efficiency  success  measure,  in  every  quarter  and 
annually.  In  contrast  Brigade  D  is  consistently  the  worst  (except  for  quarter  2  when  it  is  second 
to  last  by  this  measure). 

It  is  instructive  to  compare  the  aforementioned  measures  of  effectiveness  (viz.,  proportion 
of  mission  achieved  and  write  rate)  to  the  efficiency  classifications  obtained  from  the  DEA  to 
discern  any  patterns  that  may  be  present.  Tables  3-5  provide  summary  results  for  NFS,  PS  and 
total  contracts. 

In  each  case,  although  inefficient  battalions  can  and  do  achieve  mission  and  have 
respectable  write  rates,  this  tends  to  be  more  evident  among  efficient  battalions.  Likewise,  low 
write  rates  and  low  proportions  of  mission  achieved  are  far  more  common  among  inefficient 
battalions.  The  general  conclusion  drawn  is  that  efficiency  is  positively  related  to  but  not 
equivalent  to  effectiveness. 
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NPS  Summary  Rresults  FY 
1993 
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PS  Summary  Results  FY  1 993 
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NPS  +  PS  Summary  Results  FY  1993 
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For  comparison  purposes,  an  efficiency  analysis  was  also  conducted,  based  on  annual 
figures,  using  a  single  output,  the  sum  of  NFS  and  PS  contracts.  This  reduction  in  the  number  ot 
input-output  dimensions  plus  the  increased  discriminatory  power  of  the  new  DBA  models  halves 
the  number  of  efficient  battalions  from  14  to  7.  Table  6  shows  a  comparison  ot  the  efficient 
battalions  using  two  outputs  and  a  single  output. 

As  already  discussed  in  the  DBA  Analyses  section,  the  efficiency  (sensitivity/stability) 
alpha  values  provide  a  natural  means  for  ranking  the  battalions  (recall  positive  alpha  indicates 
efficient,  negative  indicates  inefficient).  A  comparison  of  battalion  alpha  values  and  efficiency 
rankings  under  two  outputs  and  a  single  output  is  shown  in  Table  7.  Although  the  absolute 
values  of  the  alphas  can  be  expected  to  differ  between  the  two  cases  (as  seen  here),  an  inspection 
of  the  two  rankings  shows  them  to  be  very  similar.  This  is  particularly  evident  for  the  robustly 
efficient  and  robustly  inefficient  battalions.  This  conclusion  is  further  illustrated  by  the 
individual  battalion-by-battalion  comparisons  shown  in  Table  8. 

Production  Function  Estimation  Results 

Our  intent  here  is  to  investigate  the  potential  of  one  method  (described  in  detail  in  the 
Empirical  Production  Function  Estimation  section  for  estimating  an  empirical  efficient 
production  function  for  Reserve  recruiting  and  thereby  obtain  estimates  of  "rates  of  change"  or 
elasticities  for  the  various  recruiting  resources  and  environmental  factors  in  technically  efficient 
recruiting. 

As  anticipated  (and  confirmed  in  our  applications)  the  increased  discriminatory  power  of 
our  new  (TR)  and  (GTR)  DEA  models  allows  us  to  go  directly  to  Step  3  of  the  solution 
procedure  discussed  in  the  Empirical  Production  Function  Estimation  section  of  this  paper. 
Estimation  of  the  empirical  efficient  production  function  was  performed  for  the  aggregate  year 
based  on  the  efficiency  results  of  Table  A5.  Fourteen  battalions  were  efficient.  This  limited 
number  of  observations  made  it  necessary  to  curtail  the  estimation  to  include  only  the  direct 
input  effects  (ignoring  the  interactive  effects).  Even  so,  difficulties  with  non-uniqueness  can 
still  be  expected  since  there  are  eight  parameters  to  be  estimated  -  one  for  each  input  and  output 
plus  a  multiplicative  constant  (see  the  Empirical  Production  Function  Estimation  section). 

To  illustrate  the  point,  the  basic  version  of  our  goal  program  (i.e.,  without  additional 
restrictions  applied  to  the  elasticities)  almost  fully  accounted  for  efficient  production  by 
assigning  an  elasticity  of  0.83  to  the  input  mission.  In  this  run  the  elasticity  for  unemployment 
rate  at  0.14  was  the  only  other  influential  factor.  The  recruiter  effects  appear  to  have  been 
completely  subsumed  by  the  effect  of  the  mission.  This  can  be  explained  by  observing  that  the 
efficient  battalions  as  a  group  achieved  or  exceeded  mission  so  that  relating  production  direcdy 
to  mission  was  straightforward. 

This  situation  can  be  remedied  by  applying  additional  considerations,  for  example,  to 
bound  elasticities  away  from  zero  and  to  constrain  elasticities  to  fall  within  fairly  broad  ranges 
reflecting  recruiting  experience.  However,  this  also  raises  the  question  of  whether  mission 
should  be  considered  a  valid  input  in  this  process.  In  separate  runs  excluding  mission  as  a  factor. 
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Table  6 


Efficient  Battalions  Comparison  1993 


BN 

Two  Outputs 

One  Output 

Battalion 

1 

Battalion 

2 

X 

X 

Battalion 

3 

Battalion 

4 

X 

Battalion 

5 

Battalion 

6 

X 

Battalion 

7 

Battalion 

8 

Battalion 

9 

X 

Battalion 

10 

X 

Battalion 

11 

X 

X 

Battalion 

12 

Battalion 

13 

X 

X 

Battalion 

14 

X 

X 

Battalion 

15 

Battalion 

16 

Battalion 

17 

Battalion 

18 

Battalion 

19 

Qattalion- 

20 

Battalion 

21- 

Battalion 

22 

X 

Battalion 

23 

Battalion 

24 

Battalion 

25 

Battalion 

26 

Battalion 

27 

X 

Battalion 

28 

Battalion 

29 

Battalion 

30 

Battalion 

31 

Battalion 

32 

Battalion 

33 

Battalion 

34 

Battalion 

35 

X 

Battalion 

36 

X 

X 

Battalion 

37 

X 

X 

Battalion 

38 

X 

X 

Battalion 

39 

Battalion 

40 

Battalion 

41 

Battalion 

42 

Total  Efficient  14  7 
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Table  7 


Battallion  Alpha  Value  Comparison  FY  1993 
Two  Outputs 


One  Output 


Table  8 


Battalion  Efficiency  Ranking  Comparison  FY  1 993 

Two  Outputs  One  Outputs 
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the  recruiter  and  local  advertising  effects  quickly  became  apparent.  Here  the  recruiter  elasticity 
ranged  from  0.70  to  0.76,  and  it  would  appear  that  now  the  mission  effects  are  subsumed  in  the 
recruiter  activity.  Effects  of  unemployment,  local  advertising  and  national  advertising  are 
comparable  at  approximately  0.10. 

The  constrained  goal  programming  method  is  a  sound  estimation  device.  However,  its 
accuracy  (as  with  traditional  statistical  regressions)  will  hinge  on  there  being  a  sufficient  number 
of  observations  to  alleviate  the  problem  of  non-uniqueness.  It  is  expected  that  a  much  improved 
estimation  can  be  achieved  by  moving  to  the  recruiting  company  level  of  analysis  provided 
adequate  data  are  available. 


SUMMARY  AND  RECOMMENDATIONS 

New  techniques  and  procedures  presented  here  provide  significant  improvement  over  the 
original  DEA  and  goal  programming  estimation  models  used  in  the  FAARRS-SHARE  system. 
This  tends  to  confirm  that  our  new  methodology  is  an  appropriate  way  in  which  to  model 
Reserve  recruiting,  although  several  important  data  issues  would  have  to  be  resolved  for  a  full 
scale  application.  It  also  strongly  suggests  that  modeling  efforts  of  Active  Army  recruiting 
would  also  be  enhanced  by  our  new  approach. 

The  sensitivity  technique  discussed  in  DEA  Analyses  section  has  been  demonstrated  here 
and  in  other  situations  to  be  an  important  supplemental  tool  when  analyzing  performances  using 
DEA.  Not  only  are  individual  classifications  more  thoroughly  investigated,  but  entire  groups 
within  the  data  can  be  analyzed  through  properties  of  their  DEA  residuals,  bringing  out 
information  in  the  data  that  was  undetectable  before. 

Multiple  observations  (e.g.,  quarterly)  of  battalions  provide  additional  rewards.  Since  the 
residual  (radius  of  classification  preservation)  for  a  battahon  can  be  viewed  as  a  random  variable 
(a  function  of  partially  random  data),  the  computed  value  is  merely  a  single  realization  of  this 
random  quantity.  Multiple  observations  translate  into  multiple  realizations,  and  considerable 
statistical  explorations  are  subsequently  possible,  not  least  of  which  concerns  the  expected  value 
of  an  individual  battalion's  residual.  One-sided  tests  of  hypotheses  about  the  expected  value  of 
the  residual  (i.e.,  <  0  or  >  0)  would  then  address  the  nature  of  a  battalion's  expected  performance. 
When  drawing  statistical  inferences  about  efficiency  based  upon  distributional  properties  of  the 
residual,  one  should  not  overlook  the  simplification  brought  about  by  using  a  univariate  random 
variable.  An  abundance  of  powerful  statistical  tests  and  procedures  are  therefore  available. 

The  DEA  also  provides  additional  important  information  not  immediately  relevant  to  the 
current  research  focus.  This  detailed  information  is  readily  available  as  required  for  routine 
battalion  management  including  resource  allocation  and  missioning,  and  we  recommend 
quarterly  or  semi-annual  applications  of  DEA  for  such  purposes. 

As  data  become  more  standardized  and  available  on  a  more  finely  disaggregated  basis, 
we  recommend  that  the  recruiting  company  (rather  than  the  battalion)  be  considered  as  the  new 
basic  unit  of  analysis.  In  particular,  this  is  likely  to  improve  our  estimation  procedures  for 
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production  function  elasticities.  In  future  research  it  would  also  be  desirable  to  investigate  other 
flexible  function  forms  for  the  production  function  estimation  as  well  as  alternative  means  for 
model  validation. 
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TOTAL  4962  5499  10461  5777  5015  10792  0.86  1.10  0.97  1364  3.64  4.03  7.67 


Second  Quarter  FY93 
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TOTAL  22253  21587  43840  21330  21826  43156  1.04  0.99  1.02  1288  17.27  16.76  34.03 


